Many organizations have copious amounts of data stored as part of their backend systems such as with their financial, personnel, and benefits systems. Each of these backend systems typically utilizes its own applications targeted to its purposes for storing the data and typically provides a set of its own interface for accessing the data be it for administrative or reporting purposes. Sometimes these systems are integrated within an organization for internal use; however, very rarely do these systems seamlessly integrate across organizations to provide uniform interface to relevant data to the external public. This becomes even more important when government initiatives such as open access to government data are embraced. Government organizations such as cities, counties and states have large amounts of backend data stored using a variety of backend systems. When a public person (third party) wants access to the data, a separate system needs to be put in place to access and export the data. One current approach for providing access to the data is to export the data to well-known spreadsheet applications (such as Microsoft's™ Excel) which is laborious at best.
However, as the amount of data becomes extremely large it is not pragmatic or even possible to process and view using spreadsheet programs as many have limitations on the number of “row” and “columns” of data they can process at any one time. Here “row” refers to a data item—such as employee—and “column” refers to an attribute of the data item—such as identification number, hire date, salary, or the like. Currently, in one example popular spreadsheet, this limit is set to 1,048,576 rows by 16,384 columns. This number of data items may be insufficient to process data items from a large government such as a city. Thus, the data must in these cases be distributed across multiple spreadsheets and accessed separately.
Moreover, the interfaces provided to a third party viewer via spreadsheets are limited to the raw, filtered, or sorted data or to the graphs possibly provided by the spreadsheet tools. These interfaces are typically targeted to people with knowledge of the data set and do not provide interesting and engaging ways to access the data they do not necessarily understand. In sum, the interfaces and tools cannot handle extremely large bodies of data (for example, from different backend systems) targeted to the data and do not always provide compelling or interesting ways to view the data.